Code Coverage and Quality

Part 2, Chapter 10


In this chapter, we'll add tools for checking code coverage and quality to the project.


Code Coverage

Code coverage is the measure of how much code is executed during testing. By adding code coverage to your test suite, you can find areas of your code not covered by tests.

Coverage.py is a popular tool for measuring code coverage in Python-based applications. Now, since we're using pytest, we'll integrate Coverage.py with pytest using pytest-cov.

Add pytest-cov to the requirements.txt file:

pytest-cov==3.0.0

Next, we can configure the coverage reports in a .coveragerc file. Add this file to the "project" directory, and then add the following config to exclude the tests from the coverage results and enable branch coverage measurement:

[run]
omit = tests/*
branch = True

Update the containers:

$ docker-compose up -d --build

Run the tests with coverage:

$ docker-compose exec web python -m pytest --cov="."

You should see something similar to:

=============================== test session starts ===============================
platform linux -- Python 3.10.1, pytest-6.2.5, py-1.11.0, pluggy-1.0.0
rootdir: /usr/src/app
plugins: anyio-3.4.0, cov-3.0.0
collected 6 items

tests/test_ping.py .                                                        [ 16%]
tests/test_summaries.py .....                                               [100%]

---------- coverage: platform linux, python 3.10.1-final-0 -----------
Name                     Stmts   Miss Branch BrPart  Cover
----------------------------------------------------------
app/__init__.py              0      0      0      0   100%
app/api/__init__.py          0      0      0      0   100%
app/api/crud.py             15      0      2      0   100%
app/api/ping.py              6      0      0      0   100%
app/api/summaries.py        20      0      2      0   100%
app/config.py               13      2      2      0    87%
app/db.py                   17      7      2      1    58%
app/main.py                 18      3      0      0    83%
app/models/__init__.py       0      0      0      0   100%
app/models/pydantic.py       5      0      4      0   100%
app/models/tortoise.py       9      1      2      0    91%
----------------------------------------------------------
TOTAL                      103     13     14      1    88%


================================ 6 passed in 0.56s ================================

Want to view an HTML version?

$ docker-compose exec web python -m pytest --cov="." --cov-report html

The HTML version can be viewed within the newly created "htmlcov" directory. With it, you can quickly see which parts of the code are, and are not, covered by a test.

$ open project/htmlcov/index.html

Add this directory along with the .coverage file to the .gitignore and .dockerignore files.

Just keep in mind that while code coverage is a good metric to look at, it does not measure the overall effectiveness of the test suite. In other words, having 100% coverage means that every line of code is being tested; it does not mean that the tests handle every scenario.

Just because you have 100% test coverage doesn’t mean you're testing the right things.

Code Quality

Linting is the process of checking your code for stylistic or programming errors. Although there are a number of commonly used linters for Python, we'll use Flake8 since it combines two other popular linters -- pep8 and pyflakes.

Add Flake8 to the requirements.txt file:

flake8==4.0.1

To configure Flake8, add a setup.cfg file to the "project" directory:

[flake8]
max-line-length = 119

This sets the maximum allowed line length to 119.

Update the containers:

$ docker-compose up -d --build

Run Flake8:

$ docker-compose exec web flake8 .

Were any errors found?

Correct any issues before moving on.

Next, let's add Black, which is used for formatting your code so that "code looks the same regardless of the project you're reading". This helps to speed up code reviews. "Formatting becomes transparent after a while and you can focus on the content instead."

Add the dependency to the requirements file:

black==21.12b0

Update the containers, and then run Black:

$ docker-compose up -d --build
$ docker-compose exec web black . --check

Since we used the check option, you should see the status:

would reformat app/api/ping.py
would reformat app/main.py
would reformat app/api/summaries.py
would reformat tests/test_summaries.py
Oh no! 💥 💔 💥
4 files would be reformatted, 11 files would be left unchanged.

Try running it with the diff option as well before applying the changes:

$ docker-compose exec web black . --diff

$ docker-compose exec web black .

Finally, let's add isort to the project as well to quickly sort all our imports alphabetically and automatically separate them into sections.

Again, add the dependency:

isort==5.10.1

Run it with the check-only and diff options:

$ docker-compose up -d --build
$ docker-compose exec web isort . --check-only
$ docker-compose exec web isort . --diff

Then, apply the changes:

$ docker-compose exec web isort .

Verify one last time that Flake8, Black, and isort all pass:

$ docker-compose exec web flake8 .
$ docker-compose exec web black . --check
$ docker-compose exec web isort . --check-only

Commit your code.




✓ Mark as Completed